A homological interpretation of Jantzen's sum formula 



Upendra Kulkarni 

Truman State University, Kirksville, MO 63501 (Email: kulkarni@truman.edu) 

Abstract 

For a split reductive algebraic group, this paper observes a homological interpreta- 
tion for Weyl module multiplicities in Jantzen's sum formula. This interpretation in- 
volves an Euler characteristic x built from Ext groups between integral Weyl modules. 
The new interpretation makes transparent For GL n (and conceivable for other classical 
groups) a certain invariance of Jantzen's sum formula under "Howe duality" in the sense 
of Adamovich and Rybnikov. For GL n a simple and explicit general formula is derived for 
X between an arbitrary pair of integral Weyl modules. In light of Brenti's work on certain 
-R-polynomials, this formula raises interesting questions about the possibility of relating 
Ext groups between Weyl modules to Kazhdan-Lusztig combinatorics. 



0. Introduction 

Let Gz be a split and connected reductive algebraic group scheme over Z. For a prime 
number p, G p will denote the corresponding group scheme over F p , the field of p elements. 
We will be concerned with (rational) representations of G p , in the course of which we will 
need to use Gz- We will need some background material on these topics. For all such 
standard facts see the recent edition of Jantzen's classic text [Jantzen] , where one can also 
find the original references. 

For a finite dimensional rational representation M of G p , its formal character is 

ch(M) = dim(M M )e(^) 

vex 



in Z[X] W , the Weyl group invariants in the integral group ring of the character group X 
of a split maximal torus. A central problem is to calculate formal characters of all simple 
modules, which are in bijective correspondence with their highest weights. To get to the 
issue of interest in this paper, first let L p (X) and V P (X) respectively be the simple module 
and the Weyl module (i.e., the universal highest weight module) corresponding to a given 
dominant integral weight A. Since each of the families {ch(L p (X)} and {ch(V p (X))} forms 
a basis of Z[A] , and since ch(V p (X)) is known by Weyl's character formula, the problem 
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stated above reduces to finding the multiplicities a Pt \ p defined by 

ch(L p (X))= a p ^ch(V p (p)). 

dominant /i 

There is the following well-known interpretaion of a p ,A/u as an Euler characteristic [Jantzen 
II.6.21] 

(*) a PiV = J2(-1Y dim Ext* Gp (VM,L p (X)). 

i 

(A lot more is known about this situation. Here is a sketch for completeness, though we 
will not need this information later. To begin with, a p AA = 1- Using the linkage principle, 
a p,\n = unless \i \ A. Using translation functors further, we may additionally take 
without loss of generality A and [i to be p-regular weights, provided the prime p is large 
enough to allow this. Then Lusztig's conjecture says that under a further condition on A 
(namely that it be in Jantzen's region) this multiplicity is the value at 1 of the appropriate 
Kazhdan-Lusztig polynomial for the affine Weyl group W p associated to G p . See [Jantzen 
II. 6-8] for these issues and [Jantzen II.B,C,H] for current status of Lusztig's conjecture.) 

The main purpose of this paper is twofold. (1) First, to observe a ho mo logical in- 
terpretation in the spirit of (*) for Weyl module multiplicities in Jantzen's sum formula 
[Jantzen II. 8. 19]. This formula calculates J2i>o ch{V p (A)), the sum of formal characters 
of all proper submodules V p (X) appearing in Jantzen's filtration of a Weyl module V P (X). 
The new interpretation involves an Euler characteristic x over built from Ext groups 
between integral Weyl modules. (This is to be expected since Jantzen's filtration is de- 
fined by working over Z.) The proof rests on the same idea as that behind (*), namely a 
fundamental Ext calculation due to Cline-Parshall-Scott-van der Kallen. Only here one 
uses the version over Z rather than the one over F p . An application is that for GL n the 
new interpretation makes transparent a certain invariance of Jantzen's sum formula under 
"Howe duality" in the sense of Adamovich and Rybnikov. It further suggests that a similar 
invariance may be true for some other classical groups too. The new interpretation also 
provides some evidence for the likely importance in modular theory of Ext groups over 
Gz, in particular those between integral Weyl modules. (The modular analogue of x is n °t 
so interesting for a pair of Weyl modules since it is easily seen to satisfy the orthonormal 
property.) With this in mind we will conduct additional analysis of the Euler characteristic 
X and briefly discuss some integral Ext groups. All this is done in Section 1. 

(2) Second, to calculate for GL n a simple and explicit general formula for the integral 
Euler characteristic x between an arbitrary pair of integral Weyl modules in terms of the 
associated dominant weights. This is done in Section 2. The formula is derived via a recur- 
sive procedure that employs the following ingredients. Certain Weyl nitrations from [ABT] 
corresponding to characteristic-free versions of Pieri-type rules, the skew representative 
theorem from [Kulkarnil], conjugate symmetry of Ext groups between Weyl modules from 
[AB2] and the calculation in [Kulkarni2] of Ext groups between Weyl modules for GL n 
whose dominant weights differ by a single root. 
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The GL n calculation of x gives us J2 i>0 ch(V^ (A)) in this case, which is already 
known in general thanks to Jantzen's sum formula. Still the x calculation is of additional 
interest for the following reason. The answer in Jantzen's formula is expressed as a linear 
combination of various ch(V p (fj,)), but due to the nature of this formula, it is not readily 
visible which dominant weights [i occur. Moreover, the ones that do occur may repeat 
(unless A is p-regular, see [Jantzen II. 8. 19, Remark 3]), so the multiplicities of various 
ch(V p (p)) cannot in general be read off easily from the formula. Thus, especially for small 
primes, it is interesting to have a formula for x between Weyl modules, which does give us 
these multiplicities. 



1. Jantzen's sum formula and Ext groups 

Let us proceed with the same set-up as in the introduction but the focus will shift 
to representations of G z (= Gz-modules). Given a dominant integral weight A, both the 
Weyl module V P (X) and its contravariant dual H P (X) (i.e., the dual Weyl module of largest 
weight A) are obtained from characteristic-free objects. So we have Z-free Gz-modules 
Vz(A) and H Z (X) such that V Z (X) <g> F p = V P (X) and H Z (X) <g> F p = H p (X). One has 
ch(Vz(X)) = ch(H z (X)) = ch(V p (X)). (The definition of ch(M) in the introduction applies 
equally well to Gz-modules that are free abelian groups of finite rank, the dimension being 
replaced by the rank of each weight space as an abelian group.) 

Vz(A) and H Z (X) may be realized as follows. Extend the scalars from Z to rational 
numbers Q to get the group scheme Gq. Consider Vq(A), the simple (jQ-module of highest 
weight A. Fix a vector v in the one dimensional A-weight space Vq(A)a- Among the finitely 
many Gz-stable lattices M in Vq(A) such that M n Vq(X)\ = Zv : the unique minimal 
one is Vz(A) and the unique maximal one H Z (X). For future use note that this gives a 
Gz-equivariant injection <p '■ Vz(A) <— > H Z (X). (In other words one has a nondegenerate 
bilinear form on Vz(A) (since as an abelian group H Z (X) is just the linear dual of Vz(A)) 
and this form is "contravariant" due to the way Gz-action is defined on H Z (X). See 
[Jantzen II. 8. 17]. We will not use this language here.) 

Prior to stating and proving the main results in 1.4, we will gather some general 
results regarding Ext groups (1.1), define and analyze the torsion Euler characteristic x 
(1.2) and review the setting of Jantzen's sum formula (1.3). In view of the likely but as yet 
unclear significance of integral Ext groups, some additional remarks regarding x an d Ext 
groups are offered in 1.5. Sections 1.6 discusses a connection of Jantzen's sum formula with 
Howe duality in the sense of Adamovich and Rybnikov. Finally 1.7 points out possible 
generalizations to other settings. All Gz-modules (except when scalars are extended to Q) 
will be finitely generated-equivalently, finitely generated as abelian groups. For a finitely 
generated abelian group M, M tor will denote its torsion subgroup and Mf r = M/M tor its 
largest Z-free quotient. If M is a Gz-module so are M tor and Mf r . 

1.1. Some basic results on Ext groups over G z (two fundamental Ext calculations, 
finiteness of Ext groups in general). Let us first record two important Ext calculations 
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due to Cline-Parshall-Scott-van der Kallen. 

Theorem. [CPSvdK] For dominant weights A and 

(i) Ext l Gz (V z (p),H z (\)) = unless (jjl = A and % = 0). 

(ii) Ext Gz (Vz(n), Vz(A)) = unless fx < A or (fi = A and i = 0). 
Note. Of course Hom Gz (V Z (A), V Z (A)) ~ #om Gz (V Z (A), #z(A)) ~ Z. 

The next result proves finite generation of Ext Gz (and therefore finiteness for i > 1). 
It is surely known (e.g., see [Jantzen II. 4. 10] for finite dimensionality of Ext G ), but does 
not seem to be published, so a proof is supplied. (Note. A more basic and immediate 
argument for finite generation than the one given is as follows. Use an appropriate Schur 
algebra 5 over Z, which is finitely generated, and appeal to Donkin's theorem (actually to 
its easy extension to the integral situation) asserting isomorphism of Exts with Extc z - 
Instead a different argument is given below to illustrate two techniques: use of Theorem 
1.1 (i) and a certain strategy to build up general Gz-modules.) 

Proposition. Fix finitely generated Gz-modules M and N and consider the abelian 
groups Ext Gz (M, N). 

(i) These groups are all finitely generated and are for large i. 

(ii) For i > these groups are all finite. 

(iii) HorriG z (M, N) is infinite iff the GQ-modules M <g> Q and N <g> Q have at least 
one isomorphic simple summand. 

Proof, (i) If the statement is true for M = two of the three modules in a short 
exact sequence then it is clearly true for the third. (We will say that each module is 
"obtainable" from the other two.) The same is true for N. We will show that all finitely 
generated modules are obtainable from the family {Vz(A)} as well as from {Hz(X)}- Now 
taking M = Vz(fi) and iV = Hz(X) will give (i) by Theorem 1.1 (i) quoted above. 

M is obtainable from M tor and Mf r . Now M tor has a composition series with factors 
Lp(X) for various primes p and dominant weights A. For fixed p the family {L P (X)} is well- 
known to be obtainable from either of the families {V^>(A)} and {H P (X)}, which in turn are 
respectively obtainable from the families {Vz(A)} and {Hz(X)} using multiplication by p. 
As for M/ r , it has a filtration whose factors are Gz-stable lattices P in various Vq(A). We 
have lattices of the same rank Vz(A) C P C i^z(A), resulting in torsion modules P/Vz(A) 
and Hz(X)/P- By the argument for torsion modules Mf r is also obtainable from either of 
the families {Vz(A)} and {Hz(X}}. This proves (i). 

(ii) In view of (i) clearly all Ext 1 are finite if M or N is torsion. So by looking at the 
appropriate long exact sequences derived from — > M tor — ► M — > Mf r and — > N tor — > 
N — > Nf r it suffices to take both M and to be Z-free. Now the result follows after 
extending scalars to Q (e.g., use the universal coefficient theorem [Jantzen 1.4.18] to relate 
Extc 2 and Exto Q ), where all representations become semisimple. 

Finally (iii) is clear. 
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1.2. The torsion Euler characteristic x- In view of Proposition 1.1 the following 
definition is valid. 

Definition. For finitely generated Gz-modules M and A define 

X (M, A) = \Hom Gz (M, N) tor \ JJ \Ext Gz (M, N)\^\ 

i>0 

Note. Another alternative is to simply drop the first term if the Horn is infinite. In 
fact for the purposes of this paper that would work just as well, since in all our uses of 
X, HoniGz {M, A) will be either Z-free or torsion. But in the general definition it seems 
preferable to include the "finite part" Hom,G z (M, N) tor . 

Clearly x is multiplicative on short exact sequences of finite modules in either argu- 
ment. Moreover if one of the arguments of x is finite, x is multiplicative on short exact 
sequences in the other argument. This implies that for torsion modules M and A, %(M, A) 
is always 1. (Proof. Apply HomG z {M, — ) to the exact sequence -> V z (u) A V z (u) -> 
V p (u) — > to get x(M, V p (u)) = 1. Now any finite A may be obtained from the various 
V p (u).) 

X fails to be multiplicative in general because infinite Horn groups exist. Let us sketch 
how this failure can be quantified. Apply HomG z (M, — ) or HoniG z (—, M) to a short exact 
sequence Gz-modules and take the corresponding long exact 

sequence. Replacing the three Horn terms by their torsion parts renders the long sequence 
possibly inexact in two places: the last Horn term and the first Ext 1 term. Let us make 
explicit the necessary adjustment to multiplicativity in two situations of interest. 

Example 1. (Breaking up M, A into free and torsion parts.) Apply Homc z (—, N tor ) 
to -> M tor -> M -> M fr to get X (M, N tor ) = X (M fr ,N tor ) since x(M tor , N tor ) = 1. 
Apply HomG z (—,Nf r ) to the same sequence and use HomG z (M t or, Nf r ) = to get 
X (M, N fr ) = x(M tor , N fr ) X (M fr , N fr ). Next apply Hom Gz (M, -) to -> N tor -> N -> 
Nfr — > to get a long exact sequence beginning as follows. 

-> Hom Gz (M, A tor ) -> ifom Gz (M, A) -> ifom Gz (M, N fr ) -> ^xt Gz (M, A tor ) -> • • • 

Now HomG z {M,N tor ) ~ Homa z {M,N) tor , giving an injection of free abelian groups of 
equal rank HomQ z (M, A) j r > Homa z (M, Nf r ). Calling the necessarily finite cardinality 
of the cokernel of this injection s, we clearly have x(M, A) = x(M, A tor )x(M, Nf r )s. All 
in all we have 

X(M, A) = x(M fr , N tor ) X (M tor , N fr ) X (M fr , N fr )s. 

It is easy to see that the correction factor s is really necessary. Otherwise the map 
HomG z (M, N) — > HomG z (M, Nf r ) would always be surjective, i.e., the exact sequence 
— > N tor — > A — > A/ r — > would always split (e.g., letting M = Nf r , the inverse image of 
id,N fr would give a splitting), which surely does not happen. To give a concrete example, 
it suffices to produce a non-split extension of a Z-free Gz-module by a torsion Gz-module. 
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For this take Gz = GL(Z n ). Let A 2 and D 2 respectively be the second exterior and di- 
vided powers of the defining representation Z n . One knows from [AB2, Section 9] that 
ExtQ z (A 2 , D 2 ) is Z/2Z if i = 1 and vanishes for other i. Using this in the long exact 

sequence obtained by applying Hom Gz (A 2 , -) to -> D 2 -> D 2 -> D 2 <g> Z/2Z -> yields 
that ExtQ z (A 2 , D 2 ® Z/2Z) = Z/2Z if i = 0, 1 and vanishes otherwise. In particular 
we get the desired non-split extension. See Remark 1.5.(5) below for similar elementary 
analysis of the relationships among several kinds of Ext groups in a general situation. 

Example 2. (x involving an extension of Z-free Gz-modules.) Take an exact sequence 
of Z-free Gz-modules O^N^P^R^O. Then for a Z-free Gz-module M one has 
exactly as in Example 1 

x(M,P) = x(M,N) X (M,R)s, 

where s is the cardinality of the necessarily finite cokernel of the map of free abelian 
groups Hom(M,P) — >• Hom(M,R). Clearly s = 1 unless M <g> Q, P<g> Q, R® Q all have an 
isomorphic simple summand. Similar considerations apply to such a short exact sequence 
in the first argument of x- I n the recursive algorithm to compute x between integral Weyl 
modules for GL n to be presented in the next section, the terminating stage of the algorithm 
involves exactly such pairs (M, R) which have infinite Horn groups. In this sense these 
numbers s are the source of the final numerical answers for X - 

1.3. Background on formal character for torsion modules and Jantzen's sum formula. 
We need some more machinery to be able to state and prove the main result. A convenient 
reference for all of this background material is [Jantzen II. 8]. 

Definition and basic properties of chtor ■ One can define a torsion formal character for 
a finite Gz-module M by 

ch t0 r{M) = divlM^eifj,), 
vex 

where div stands for taking the divisor of a rational number (here an integer). (Compare 
the definition of v c in [Jantzen II. 8. 12].) Clearly ch tor (L p (X)) = ch(L p (\))\p] and ch tor 
is additive on short exact sequences of finite Gz-modules. As A varies over all dominant 
weights and p over all primes, each of the two families {chtor(L p (X))} and {c^ or (V^(A)) = 
chtor (H P (X))} forms a basis of the abelian group of all torsion formal characters. Note 
that since ch(V p (X)) = ch(Vz(X)) is independent of p, it makes sense to speak about 
the coefficient of ch(Vz(X)) in ch tor (M), this coefficient being the divisor of a unique 
positive rational number (that will be 1 for all but finitely many A). We will need these 
considerations while explaining the setting of Jantzen' s sum formula, which will be our 
next task. 

Jantzen 's filtration and Jantzen's sum formula. Recall the injection <p '■ V"z(A) <— >■ 
Hz(X). Jatzen's filtration is a descending filtration V^(A) of V P (X) defined as follows. 
Fixing p for the moment, first let V Z (X) be the submodule 4>~ 1 (p l Hz(X)) of Vz(A) and then 
Vp(X) = the image of V Z (X) under the canonical map Vz(A) — > V P (X). (So in particular 
V p (X)/Vp (A) = the image of (p <g> idF p , which is well-known to be L P (X).) 
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Set Q(X) = coker ((f)). By a well-known argument due to Jantzen, 

ch tor Q(\) = j2(T, ch ( v i( x )n w- 

p \i>0 / 

At the same time it is immediate (e.g., after diagonalizing <p by using suitable bases for 
Vz(A) and Hz(X)) that calculating cht or Q(X) is equivalent to calculating the determinant 
of 4> on each weight space. In most cases Jantzen succeeded in calculating the determinant 
(more precisely its p-adic valuation when p is not small and without restriction for type 
A) and so proved a formula for J2i>o ch(V p (A)), which was soon afterwards obtained in 
general by Andersen via a different method. See [Jantzen II. 8] for a detailed explanation 
of all this and the formula itself. We do not need the formula here. Rather, our goal is to 
observe an Euler characteristic interpretation for the Weyl module multiplicities occuring 
in this formula. These multiplicities, denoted 6 p ,a^, are defined by 

i>0 dominant fi 

Note that only fx < X can have nonzero coefficients on the right hand side. 

1.4. A formula for ch tor using \ and application to the sum formula. We will inter- 
pret the coefficients & p ,a^ i n terms of x between integral Weyl modules. This will be a 
consequence of the following integral analogue of (*) in the introduction (more precisely, 
analogue of [Jantzen II.6.21]). 

Proposition. For a finite Gz-module M, 

ch tor (M) = <tiv(x(Vz(lJ.),M))ch(V z (v,)) 

dominant fx 

= J2 -*v(x(M,H z (/jL)))ch(V x (/JL)). 

dominant /i 

Further, the statements stay true if one replaces Vz(/i) or Hz(fi) by any Gz-stable lattice 
inside Vq(ju). 

Proof. By additivity of chtor and of the divisor of x(Vz(/u), — ) on finite Gz-modules 
it is enough to check the first equality for M = H p (y) for each prime p and each dominant 
integral weight v. One has chtor(H p (u)) = ch(Vz(v))\p]. For the other calculation apply 

Homc z (Vz(p), — ) to — > Hziy) — ► Hz{v) — > H p (v) — > and use Theorem 1.1 (i) above 
to get x(Vz(/u), H p (v)) = p 5 ^ . The second equality is checked similarly using M = V p (u). 
Since x is 1 for a pair of finite Gz-modules, using multiplicativity of x(— , M) (respectively, 
of x(M, — )) one may replace Vz(/i) (respectively i?z (/•*)) inside x by any Gz-stable lattice 
inside Vq(/j,), thus proving the last statement. 
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Corollary. (Homological interpretation of Jantzen's sum formula.) The multiplicities 
of c/i(Vz(aO) m Jantzen's sum formulas for V P (X) for various primes p are given by 

E^AmH = -div(x(V z (ji),V z (\)). 
p 

Proof. Note that 

ch tor Q{\) = W E b P Mch(Vz(»)) \p}= E ( E^vW ) ch(VzQj)) 

p \ dominant /j / dominant ji \ p / 

= E di«(x(^zO*),Q(A)))cfc(y z O*))- 

dominant /x 

Here the first line comes simply from the set-up in 1.3 and the second from the first equality 
in Proposition 1.4. So all we need to show is that 

div(x(V z (»),Q(\))) = -div(x(V z (n),V z (\))). 

For this take the long exact sequence obtained by applying Homc z (Vz(/u), — ) to — > 
Vz(A) -> i^z(A) -> Q(A) -> 0. Now x(V z (ij),H z (\)) = 1 by using Theorem l.l(i) once 
again and hence we have the desired result. (Note that the first two Horn terms in the 
long exact sequence-potentially the only infinite ones-vanish unless fj, = X. If fx = A, 
these terms are Z and the map between them is an isomorphism. So multiplicativity of 
X (V z (li), — ) is valid in either case. The latter case is trivial anyway, since then all terms 
other than the first two vanish and all three x's are 1.) 

1.5. Complementary remarks on x an d on Ext groups. (1) Note that the analogue 
over F p of x between Weyl modules is uninteresting as it satisfies the orthonormal property. 
One has ^(-^dim Ext% p (V p (\), V p (fi)) = S Xfl by [Jantzen II.6.21]. 

(2) x f or some other pairs of modules. One may contemplate x(A B) where A and 
B are Gz-modules from the three families {Vz(A)}, {H z {\)} and {Q(X)}. Using informa- 
tion obtained so far, it is straightforward to analyze the nine possibilities. By Theorem 
l.l(i), x(Vz(/i), H Z (X)) = 1. Q(X)) = 1 since both arguments are torsion mod- 
ules. x(Vz(/j),Vz(X)) = x(H z (X),H z (li)) (by contravariant duality) = 1/x(Vz(aO, Q(A)) 
(by Theorem l.l(i)) = X (Q(>),H z (ji)) = l/x(H z (/i),Q(X)) = X (Q(X), V z (p)) (last three 
expressions by Proposition 1.4). This leaves X (H z (/i), V Z (X)). To relate this to previous 
cases apply Homo z (—,Vz(X)) to — > V z (p) — > H z (/S) — > Q(fS) — > and use known in- 
formation to get X (H z (n), Vz(A)) = X {V z (fj,), V z (X))x{V z (X), V z {ll)). The symmetry in A 
and fi is to be expected in view of contravariant duality. Note that by Theorem 1.1 (ii) at 
most one of the two factors on the right hand side may be different from 1. 

(3) A certain skew- symmetry of x- It follows from Proposition 1.4 that for a torsion 
Gz-module M, X (M, V z (fj,))x(V z (iJ,), M) = 1. Using multiplicativity of X (M, -) as well as 
of X (— , M), and since x is 1 for a pair of torsion modules, one may replace V z (ji) by any 
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Gz-module N via a route similar to the one followed in the proof of Proposition l.l(i). 
So for a torsion Gz-niodule M and for any Gz-module N, one has %(M, N)x{N, M) = 1. 
This is not true in general if M is not torsion, e.g., when M and N are both integral Weyl 
modules in view of Corollary 1.4 and Theorem 1.1(h). 

(4) Of course it is much more interesting (and harder) to calculate the Ext groups 
themselves rather than merely calculating \. Here is a reason why the Ext groups be- 
tween integral Weyl modules are likely to be important. Note that we made essential use 
of Theorem 1.1 (i) only "up to Euler characteristic," not in its full strength. This theo- 
rem gives Exi l Q^(Vz(p),Vz(X)) — Ext Gz (Vz{ti),Q{X)) and clearly the structure of Q(X) 
is intimately related to Jantzen's filtration itself. It would be very interesting if one can 
relate individual Ext groups between integral Weyl modules to Jantzen's filtration. It 
seems reasonable to speculate that Kazhdan-Lusztig type combinatorics should come into 
play in such a relationship. (There is already a hint to this effect for type A. See Re- 
mark 2 after the proof of Theorem 2.3.) For instance one can ask the following question. 
Consider the alternating sum obtained by taking the divisor of the defining expression for 
x(^z(aO, Vz(A)). What is the polynomial obtained by replacing —1 in this expression by 
an indeterminate ql (Unlike over F p , this is a weaker question than knowing the individual 
Ext groups since in general the structure of an abelian group is not determined by its size.) 

(5) Elementary observations relating Ext Gz and Exta p . Consider a G p -module L 
and a Z-free Gz-module M. First we have the isomorphism of Fp-vector spaces (compare 
[McNinch, Lemma 3.1.1b]) Ext Gz (M,L) ~ Ext Gp (M <g> F p , L). Proof. Using * to denote 
the linear dual over Z, [Jantzen 1.4.2(1) and 1.4.4] give Ext i Gz (M, L) ~ F(G Z , M*®L) and 
Ext % G (M®Fp, L) ~ H l (G p , M*<S)F p <S)L). Now the Hochschild complexes computing both 
group cohomologies are isomorphic. Alternatively, take a projective resolution P. — > M 
over an appropriate Schur algebra Sz- Tensoring by F p gives a projective resolution of 
M ® F p over the Schur algebra S p = Sz <8> F p . Applying the appropriate Horn to each 
resolution gives isomorphic complexes computing the required Ext groups. 

Next, it is easy to relate these isomorphic Ext groups to Ext l Gz (M <g> F p ,L). Apply 

Hom Gz (—, L) to the short exact sequence 0-^M-^M^M®F p ^0. The resulting 
long exact sequence easily breaks up into short exact sequences, giving the isomorphism 
of Fp-vector spaces Ext Gz {M <g> F p , L) ~ Ext%~^(M, L) © Ext Gz (M, L). 

One can say more in the following special situation. Let L = N <S> F p , where N is a 
Z-free Gz-module. Suppose the localization at prime p of the abelian group Ext l Gz (M, N) 

is a direct sum of cyclic groups. Using the short exact sequences — > M M — > 
M ® Fp — > and 0^A^^A^^A^®Fp^0asin the previous paragraph, one gets 
the following equalities, dim Ext Gz (M <g> F p , N) = n^i + n^. dim Ext Gz (M, N <g> F p ) = 
dim Ext G (M ® F p , <8> F p ) = rii + n i+ i. (The preceding equality can also be seen from 

the Universal Coefficient Theorem [Jantzen 1. 4. 18a].) dim Ext Gz (M <g> F p , N ® F p ) = 
rii-i + 2ni + n i+1 . 

The foregoing considerations may be of particular interest in two cases because of the 
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significance of the Ext groups involved. Take M = V z (fx) in each case and in turn let 
L = L p (X) (see (*) in the Introduction) or L = V P (X) (in view of the previous remark). 

1.6. Symmetry of sum formula under Howe duality and complements. Consider Young 
diagrams of two partitions A and \i. These may be considered to be two dominant weights 
for GL(F), where F is a free abelian group of rank at least as much as the number of rows 
in A as well as fx. Let A (respectively, fx) be the Young diagram obtained by transposing 
the rows and columns of A (respectively, fx). X, fx are dominant weights for GL(E), where 
E is a free abelian group of rank at least as much as the number of rows in A as well as 
fx. (One could also work with a single group of large enough rank, or in the category of 
polynomial functors without having to worry about the rank at all.) Now the functor O 
from [AB2, Section 7] (a characteristic-free form of Howe duality, in the sense similar to 
[AR]) combined with contravariant duality gives the following "conjugate symmetry" of 
Ext groups between Weyl modules. 

(1.6.1) Ext GL{F) (V z (fx), V Z (X)) ~ Exth L{E) (y x {\), V z (f)). 

Still working with GL{F), a more elementary symmetry of these Ext groups is as follows. 
Enclose Young diagrams of A and fx in a rectangle of height rank(F) and of suitable width 
w. Let A c and fi c respectively be the partitions obtained by taking the complements of A 
and \x within this rectangle. One has (e.g., by [ABW, II. 4]) Vz(fx) = Hz(fx c )* <8> (det) w and 
likewise for A. (Here * denotes ordinary linear dual of a representation, where one uses 
the group antiautomorphism taking g to g~ x to convert the natural right action into a left 
one.) Now 

Ext GL{F) (V z (Li),V z (X)) ~ Exf GLiF) (H z (fu c y <g> (det) w ,H z (X c y <g> (det) w ) 

^Exf GL{F) {H z {fu c )\H z {Xy) 
^Ext GL{F) (H z (X c ),H z (fx c )) 
~ Ext GL(F) (V z (fu c ),V z (X c )). 

Here the second line comes from canceling the determinant, the third by using linear 
duality and the last by using contravariant duality. 

It follows immediately from (1.6.1), (1.6.2) and Corollary 1.4 that for general linear 
groups Jantzen's sum formula is stable under Howe duality and under complements. More 
precisely one has the following. 

Corollary. For the general linear groups (in the terminology of 1.3), 

Adamovich and Rybnikov [AR] have constructed Howe duality functors in positive 
characteristic for several other pairs of classical groups. Their set-up gives an isomor- 
phim of Ext groups analogous to a combination of (1.6.1) and (1.6.2), but in character- 
istic p. It is not clear that their isomorphism is valid while working over Z. The groups 
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Ext l Gz (Vz(fi) , Vz(A)) for all i determine the corresponding modular Ext groups (via, e.g., 
[Jantzen 1.4. 18a]), but not vice versa. Still the work of Adamovich and Rybnikov makes it 
reasonable to ask whether Jantzen's sum formulas are stable under Howe duality for any 
other pairs of groups besides the general linear groups. This should (at least in principle) 
be verifiable directly from the known sum formulas. 

Notes. (1) [McNinch] proves the following intersting connection within this cluster of 
ideas. Howe duality in [AR] carries Jantzen's nitrations to Andersen's "tilting nitrations." 
(2) Here is yet another connection between Andersen's tilting filtartions and Jantzen's 
nitrations. (This is entirely independent of Howe duality, but is still mentioned here for 
completeness.) Using the results in this section, it is possible to derive Andersen's "titling 
sum formula" as a formal consequence of Jantzen's sum formula. Currently this is known 
only if the characteristic is not small, since the original proof uses regular weights. The 
new proof will be explained in [Kulkarni3] . 

1.7. Generalization to highest weight categories. The arguments in this section used 
certain objects (such as simple, Weyl and dual Weyl modules indexed by a suitable partially 
ordered set) and facts about these objects (such as suitable vanishing properties) that are 
available in other situations as well. The natural home for the needed objects and vanishing 
results is the axiomatic notion of a highest weight category (= representations of a quasi- 
hereditary algebra) due to Cline-Parshall-Scott. Additionally one needs a suitable model 
of such a category over a principal ideal domain from which the category of interest is 
obtained by reduction modulo a prime. See for example the set-up in [McNinch] . It should 
be straighforward to carry over the results in this section (except 1.6) to such an axiomatic 
setting. In particular analogues of the main results should hold in the following situations: 
the BGG cateory O (using the version constructed by Gabber- Joseph) and representations 
of quantum groups at a root of unity. (One has Jantzen's nitrations and sum formulas in 
these situations as well.) Because of the focused nature of interest in this paper, we will 
not carry out any of these extensions here. 



2. A formula for x(V"z(/u), Vz(A)) for the general linear group 

Henceforth let Gz = GL(F), where F is a free abelian group of finite rank. The 
rank of F will be essentially immaterial (see below for the precise statement). We will 
calculate x(Vz(/i), Vz(A)) for arbitrary dominant weights A and fi and so obtain for this 
group a version of Jantzen's sum formula (in view of Corollary 1.4). Since the argument 
has some combinatorial intricacy, it will be broken up as follows. In 2.1 we will develop an 
algorithm to calculate the desired x- The algorithm will be illustrated in 2.2 by working 
out two simple classes of examples, after which we will derive the general formula in 2.3. 
But first let us (slightly) reformulate the problem so as to arrive at the setting in which 
the algorithm will take place. 

Each of the dominant weights A and \i may be identified with a weakly decreasing set 
of rank(F) integers. By tensoring with the determinant enough times, we may take these 
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integers to be nonnegative without loss of generality. Thus A and fi may be taken to be 
just partitions with at most rank(F) rows. We will freely identify partitions with their 
Young diagrams. 

Now an arbitary partition may be considered a dominant weight for any GL(Z n ) 
such that n > the number of rows in that partition. As soon as this condition on 
n is met for both A and //, x(V"z(/u), Vz(A)) is independent of n for the following rea- 
son. Let N > n > the number of rows in A as well as those in fi. For the moment 
let A;v (respectively A n ) denote the dominant weight corresponding to the partition A 
for the group GL(Z N ) (respectively GL(Z n )) and likewise define fi n ,fi^. Then one has 
Ext GL(Zn) (V z (fi n ), V z (\n)) - Ext l GL{zN) (V z (lJ,N),Vz(\N)) by, e.g., [Kulkarni2, Proposi- 
tion 1.1]. So henceforth we will deal with pairs of arbitrary partitions, always assuming 
that we are working over an appropriate GL(F), i.e., one for which rank(F) is large enough 
for the partitions in question to be valid weights. 

2.1. An algorithm to compute xO^z (/•*)> Vz(ty) for GL(F). Let us begin by setting up 
an induction and by making some simple reductions. 

(1) If n ft A then x(Vz(aO, Vz(A)) = 1 by Theorem 1.1(h). Therefore we may assume 
that fi < A and we may further induct on the number of steps by which \i and A differ 
in (any linearization of) the dominance partial order. The base case of this induction is 
known due to [Kulkarni2, Theorem 2.1] and is subsumed in (2.1.3) below. 

(2) Note that fi < A means in particular that A and fx have the same degree, i.e., the 
same number of boxes. (Of course the necessity of this condition for nontriviality of x is 
clear even before by considering the action of the center of GL{F).) Let us induct on this 
degree as well. In degree 1 there is only one Weyl module, namely F, and x(F, F) is 1 by 
either part of Theorem 1.1. 

(3) If the first rows (or columns) of fi and A have the same length then by [Kulkarni2, 
Proposition 1.2] we may strip them off without affecting the Ext groups and appeal to 
induction. So we may assume that fi and A differ in the lengths of their first rows and also 
in the lengths of their first columns. 

Now let us proceed to the main inductive step. Let A' be the partition obtained by 
removing the first column of A and call the length of this column t. By reductions (1) 
and (3), the first column of \x is longer than that of A. Removing the first t boxes in this 
column gives the skew partition fi/l l . The partition 1* is just a single column of length 
t and the corresponding Weyl module is A t (F) = the t-th exterior power of the defining 
representation F. A t (F) is also the dual Weyl module corresponding to 1*. Consider the 
following equality given by the Skew Representative Theorem from [Kulkarnil]. 

(2.1.1) x(Vz(fu), A*(F) ® Vz(A')) = x(VzO*/l*), Vz(A')), 

where Vz(/u/P) is the skew Weyl module corresponding to fi/l 1 defined in [ABW], where 
it is denoted K fJi / 1 t(F). Let us analyze both sides of the equality (2.1.1) using the almost 
multiplicativity of x- 
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On the right hand side, Vz(/u/l*) has a Weyl filtration with factors V z (fj}), . . . , V z (/j k ) 
where \i % are all the partitions contained in (j, such that \ij 1 ii l consists of t boxes no two of 
which are in the same row. In other words, [i l are all the partitions obtainable by removing 
t boxes from the rightmost border strip of 11. This is the characteristic-free skew Pieri rule 
from [AB1, Section 3], where one can also find a more formal statement and a proof in the 
contravariant dual case of Schur modules, which carries over easily to ours. By induction 
on the degree x(Vz(/U l ), Vz(A')) are all known. 

By the characteristic- free Pieri rule in [AB1, Section 3], A*(F) <g> Vz(A') has a Weyl 
filtration with factors V^A 1 ), Vz(A 2 ), . . . , Vz(A m ), where A- 7 are all the partitions contain- 
ing A' such that A- 7 /A' consists of t boxes no two of which are in the same row. Clearly 
these include A = A 1 (say), and A > X J for j > 1. So by induction on the dominance order 
x(Vz(aO? Vz(A j )) are all known for j > 1. 

Case 1. If Hom(Vz(fi), A t (F) <g> Vz(A')) ~ Hom(V z (ji/l*), V Z {X')) is zero, i.e., if none 
of the A- 7 equals fx, i.e., if none of the \i % equals A', then x is multiplicative as we glue the 
V Z (X J ) together to get A t (F) ® Vz(A') (respectively, the Vz(fi l ) to get Vz(/u/l*)) because 
all the Horn terms in the associated long exact sequences vanish. Therefore we get the 
following equations. 

x(VzM,A'(F) ®Vz(A')) =Hx(Vz(v),V z (V)). 

(2.1.2) 3 

x(Vz(/*/l*), V Z (X')) = X {Vzfr% V Z (X% 

i 

Now one easily finds x(Vz(ji), V Z (X)) from (2.1.1) and (2.1.2). 

Case 2. If Case 1 does not happen then it is clear from the Pieri rules that x f ans to 
be multiplicative at exctly one stage in gluing. By the work done in Example 2 in 1.2, the 
equations (2.1.2) now need to be modified by writing certain correction factors on their 
right hand sides. At first glance this seems to make it necessary to compute these integers 
every time we are in Case 2. The only way I know of directly doing this computation 
involves finding explicit generators of certain Horn groups. This is theoretically possible, 
but quite hard in practice, even in the relatively simple case treated in [Kulkarni2, Theorem 
2.1], as seen in the proofs of Lemmas A-C there. 

Fortunately we can wriggle out of this difficulty in most cases by analyzing the combi- 
natorics of Pieri's rule and by using conjugate symmetry of Ext groups (1.6.1). Vizualizing 
the dominant weights occuring in A t (F)<S>V z (X / ) reveals that if one encounters Case 2 then 
fi must be obtainable from A by removing some of the boxes from the rightmost border 
strip of A and placing them at the bottom of the first column. 

Now (1.6.1) gives x(V"z(/u), Vz(A)) = x(V"z(/i), V z {\)). So if we encounter Case 2, we 
may discard the pair (fx, A) of dominant weights and work with the pair (A, fi) instead. 
A little further thought shows that the only way (A, /}) will also lead to Case 2 is when 
\x has a single box in its last row and A is obtained by removing this box and placing 
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it at the end of the first row. (Recall that by the reduction (3), p and A must differ in 
their first rows as well as in their first columns.) So A — p = a positive root a. But this 
is precisely the situation treated by [Kulkarni2, Theorem 2.1]. This theorem gives the 
following calculation of all Ext groups for such pairs of Weyl modules. Ext 1 is cyclic of 
order (p + p, a) + 1 = hook length of the box in the first row and first column of p (or A) , 
and all other Ext groups vanish. So for A = p + a one has 

(2.1.3) x(Vz(jjl),V z (X)) ' 



(p + p, a) + 1 

This ends the recursion and completes the description of the algorithm. 

Remarks. (1) For future use note the following extra flexibility that may be built into 
the above algorithm, t may be chosen to be the length of any column in A. Then A' would 
be the partition obtained by deleting the rightmost box in each of the top t rows of A. The 
induction goes through just as before with the following differences. For one, different sets 
of p 1 and A J are involved in the computation. Moreover the characteriztion of when one 
encounters Case 2 is no longer as clean. Nor is the escape by use of conjugate symmetry 
guaranteed if one is required to use a given t. But of course one may then use a different t 
(e.g., just follow the algorithm above). The point is that sometimes choosing a different t 
will make the calculation simpler. For example choosing t = 1 (of course the last column 
of A must be of length 1 for that) may considerably simplify the combinatorics for large 
partitions. Note that [Kulkarni2, Theorem 2.1] was proved by taking t = 1. In the proof 
of that theorem recursion could be controlled sufficiently to give all Ext groups, not just 
X- 

(2) (Relating the algorithm to symmetries of Ext groups between Weyl modules.) By 
(1.6.1), (1.6.2) and using the terminology there we have 

X( W), ^z(A c )) = x(V z (p), V Z (X)) = X (Vz(A), V z {p)). 

Under the isomorphism (1.6.2), using the algorithm for the pair of partitions (p,X) by 
splitting off the first column of A is seen to be tantamount to using the algorithm for the 
pair (p c , A c ) by splitting off the last column of A c . Similarly one can devise a version of 
the algorithm that is consistent with the second equality. In place of (2.1.1) this version 
will rely on a procedure that splits off the top row of p rather than the first column of A. 
Later both of these symmetries will be very useful for substantial reductions while proving 
a general formula for %(Vz(/i), V Z (X)). 

2.2. Examples. Before stating and proving the formula for x f° r a general pair of par- 
titions, it will be instructive to see the above algorithm in action in some simple examples. 

Example 1. (GL 2 .) Let us compute x(V"z(/u), V Z (X)), where p and A are partitions 
with at most two parts. By the initial reductions the general case immediately reduces to 
the following: p = (a, 6), A = (a + 6, 0), where a > b > 0. For convenience let us denote 
xC^z (/•*), Vz(A)) by x[a, 6]. Clearly the only choice for t here is t = 1. Unless 6=1, the 
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algorithm does not lead to Case 2. Using (2.1.1), (2.1.2) and reduction (3) one gets the 
following equations. 



X[a, b] x[a -1,6-1] = x[a, 6-1] x[a -1,6] if a > 6. 
X[a, a] x[a - 1, a - 1] = x[a, a -I]. 



If 6 = 1 one has %[a, 1] = Now one easily derives that 



x[M] 



b 



a + 1 



Example 2. (x f° r two hook partitions.) Let \x = (a, l b ), A = (a + s, l 6_s ), i.e., both 
diagrams have the shape of a hook. For convenience let us denote x(Vz(/f), Vz(A)) by 
x[ct, 6, s]. As in the algorithm, let us choose t = 6 + 1 — s, i.e. the length of the first column 
of A (The reason for this choice will be clear soon.) If s ^ 1, the algorithm does not lead 
to Case 2. In this case one gets the following equation using (2.1.1) and (2.1.2). 



Notice that for this calculation, we only needed to deal with pairs of hooks i.e., no other 
partitions appeared in recursion. A little thought reveals that if we had chosen t = 1, this 
would not have been the case and the computation would have been much more unwieldy. 

2.3. A formula for %(Vz(/i), Vz(A)). It turns out that there is a fairly simple formula 
for x(Vz(/i), Vz(A)) in terms of the geometry of the diagrams of the involved partitions. 
Once the formula is guessed (after working out several classes of examples like the ones 
shown above), it is not so hard to prove by judicious use of the algorithm and careful 
bookkeeping. Before stating the final answer in the next theorem, let us set up some 
terminology. A skew partition is connected if a rook can go from any box in it to any 
other by ordinary chess moves. A skew hook (also called ribbon or a border strip) is a 
skew partition not containing a 2 by 2 square. We may call a skew partition failing this 
condition overconnected. Thus a skew partition will fail to be a connected skew hook by 
being disconnected or overconnected or both. We could call a connected skew hook a 
snake. The "right endpoint" (respectively, "left endpoint" ) of a skew partition will mean 
the rightmost box in its top row (respctively, the leftmost box in its bottom row). This 
terminology will be used mainly when the skew partition is a skew hook. 

Theorem. Let fx and A be arbitrary partitions. Considering \i and A as sets of appro- 
priately situated boxes in a plane, let v be the partition A D fi. We will say that fi and A 
differ by connected skew hooks if the skew partitions \xjv and \jv are both connected skew 



X[a, 6, s] x[a, 6, s - 1] = x[a, s - 1, s - 1] xW ~ !» s > 4 



If s = 1 one has x[a, 6, 1] = ^W. Now one easily derives that 
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hooks. Then one has the following. (1) x(Vz(/i), Vz(A)) is 1 unless fx < A and moreover fx 
and A differ by connected skew hooks. (2) If fi and A do differ by connected skew hooks 
and li < A as well, then 



where the symbols have meanings as follows. £ is the the equal number of boxes in skew 
hooks fijv and Xjv. We will call this the length of the skew hook, not to be confused with 
the number of rows in the skew hook, d is the total number of right and up moves that 
each box in fijv has to make (i.e., the distance this snake has to cover by sliding along 
the border of the diagram) in order to transform fx into A. r is the sum of the number of 
nonempty rows in fijv and that in Xjv. By drawing pictures, the numbers £ and d can 
also be seen to be the following hook lengths. £ is the hook length of the box in A that is 
in the same row as the right endpoint of A/z/ and the same column as the left endpoint of 
A/z/. The previous sentence stays valid after replacing each A by fx. d is the hook length 
of the box in fx that is in the same row as the left endpoint of Xjv and the same column 
as the left endpoint of fijv. d is also the hook length of the box in A that is in the same 
row as the right endpoint of Xjv and the same column as the right endpoint of fijv. 

Proof. Recall the following from 2.1. By Theorem 1.1(h) we may assume that fx < X. 
We may also assume that A has a strictly longer first row and strictly shorter first column 
than fx. So the entire last column of A is missing in fx and the entire last row of fx is missing 
in A. We will induct on the equal number of boxes in fx and A and on the number of steps 
by which A and fx differ under (a linearization of) the dominance partial order. Note that 
the base case of each induction is already known. By (2.1.3) we may additionally assume 
that A/z/ (and fxjv) contains more than one box. We will follow the algorithm as extended 
in the first remark following it. More precisely we will always use t = the length of the last 
column of A. It will be clear that the difficulty with infinite Horn encountered in Case 2 
in 2.1 never arises (partly due to the use of (2.1.3) at the outset). 

The proof will consist of a series of reductions and one crucial calculation requiring 
treatment of several cases. Throughout it will be easier to follow the combinatorial ar- 
guments by visualizing the diagrams of A and fx with A/z/ and fi/u (the set differences 
between A and fx) "colored" differently from v. 

Step 1. (Getting rid of almost all overconnectedness.) Suppose that A/z/ contains 
a 2 by 2 square (i.e., is overconnected) and that A has at least one column strictly to 
the right of this 2 by 2 square. When one runs the main step of the algorithm with t = 
the length of such a column (say the last one), for each resulting pair (fx, A J ) the skew 
partition A J /(A J n fx) will stay overconnected. This is simply because A J is obtained by 
moving down some boxes to the right of the "excess 2 by 2 square," which will clearly 
keep this square intact. Similarly for each pair (fx l ,X') the skew partition A'/(A' n fx 1 ) 
will also stay overconnected. Now equations (2.1.1), (2.1.2) and induction give the desired 
triviality of x(V"z(/u), Vz(X)). By conjugate symmetry (1.6.1), one also gets the triviality 
of x(^z(aO, Vz(X)) when fijv contains a 2 by 2 square and fx has at least one row strictly 
below this 2 by 2 square. 
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Enclose \i and A in a rectangle and recall the symmetry of Ext under complements 
(1.6.2). Since any 2 by 2 square in X/v is present is A and absent in fi, the same square 
will be present in fi c and absent in A c . If such an excess 2 by 2 square occurs in A and is 
situated below the top row of A, the corresponding excess square in fi c must be situated 
above the bottom row of \i c . Now applying the last sentence of the previous paragraph to 
the pair (fi c , X c ) and using (1.6.2) we get that %(Vz(/i), Vz(A)) = 1. Conjugate symmetry 
(1.6.1) gives the same conclusion if fi/v contains a 2 by 2 square that is situated to the 
right of the first column of fi. 

By the previous two paragraphs we may assume the following. Xjv contains at most 
one 2 by 2 square and such a square must occur in the last two columns and the first two 
rows of A. Similarly fi/v contains at most one 2 by 2 square and such a square must occur 
in the first two columns and the last two rows of fi. Henceforth these assumptions will be 
in force throughout the proof. 

Step 2. (The crucial calculation.) Suppose the last column of A (which is absent from 
fj, by our reduction) consists of exactly one box. We will run the algorithm with t = 1 
and show by induction that the claimed formula holds. By Step 1, X/v is a (possibly 
disconnected) skew hook. We will treat three cases. 

Case A. Suppose X/v and fi/v are both connected skew hooks. We will analyze both 
equations in (2.1.2). In the first equation it is easy to see by induction that there are 
at most three partitions A- 7 other than A that lead to nontrivial x(Vz(/u), Vz(A J )). These 
are described in cases (A.l.i) through (A.l.iii) below. For ease of description let the right 
endpoint of fi/v be in •u-th row and v-th column of fi. Let the left endpoint of X/v is in 
x-th row and y-th column of A. 

(A.l.i) When A- 7 is obtained by adding a box to A' immediately below the left endpoint 
of X/v, i.e., in (a; + l)-th row and y-th column. This gives a valid partition precisely when 
the x-th and (x + l)-th rows of v are of equal length y — 1. In that case A- 7 fl fi = v 
and moreover A- 7 and fx still differ by connected skew hooks of length I. But the distance 
needed to slide one of these skew hooks into another is one less than before and A J /z/ has 
one more row than X/v. By induction 



(A.l.ii) When A J is obtained by adding a box to A' in the position of the left endpoint 
of fi/v. This gives a valid partition precisely when fi has exactly one row more than A. In 
a manner similar to (A.l.i) it is easy to see that in this case we have 



(A.l.iii) When A J is obtained by adding a box to A' in the position of the right endpoint 
of /i/v, i.e., in -u-th row and v-th column. This procedure gives a valid partition precisely 
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when in \xjv this right endpoint is the only box in its row, i.e., when the -u-th row of v has 
length v — 1. In a manner similar to (A.l.i) and (A.l.ii) it is easy to see that in this case 
we have 



The analysis of the second equation is very similar. Here we need to look for the // 
that will give nontrivial x(Vz(/i l ), Vz(A')). There are at most three possibilities, which are 
listed below. 

(A.2.i) When n % is obtained from \i by removing the box immediately to the left of 
the left endpoint of X/u, i.e., the one in x-th row and (y — l)-th column. This gives a valid 
partition precisely when x-th and (x + l)-th rows of v are of unequal legnths, i.e., when 
(A.l.i) is not possible. In that case A' and ji l still differ by connected skew hooks of length 
£; the distance needed to slide one of these skew hooks into another is one less than before; 
and the number of rows in each skew hook stays unchanged. By induction 



(A.2.ii) When \i % is obtained from ji by removing the box at the left endpoint of /i/v. 
It is easy to see that this gives a valid partition precisely when (A.l.ii) is not possible. 
Moreover, in that case %(Vz(//), Vz(A')) is the reciprocal of the number obtained in (A.l.ii). 

(A. 2. hi) When \i l is obtained from \i by removing the box at the right endpoint of 
[ijv. Again this gives a valid partition precisely when (A.l.iii) is not possible. In that case 
x(Vz(/J> 1 ), Vz(A')) is the reciprocal of the number obtained in (A.l.iii). 

Note that a fourth \i % seems possible at first glance. Namely one could try to take off a 
box from \i so as to somehow augment the skew hook Xjv at its right endpoint (similar to 
the way it was augmented on the left in (A.2.i)). But this is clearly seen to be impossible. 
We will encounter such a possibility later in Case C. 

So we may as well suppose that none of the cases (A.2.i) through (A. 2. hi) occurs and 
thus all of the cases (A.l.i) though (A.l.iii) do. The desired claim is now immediate from 
(2.1.1) and the following trivial calculation. 



Case B. Suppose A/z/ is a connected skew hook but [xjv is not a connected skew hook. 
Proceeding in a manner similar to Case A, it is easy to see that there are only two ways in 
which one of the equations in (2.1.2) could yield a lower term with nontrivial x- (If every 
lower x m both equations is trivial, the desired result, namely triviality of x(V"z(//), Vz(A)), 
is immediate.) 



i) 



7 1 
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(B.l) If li/v contains a 2 by 2 square (by Step 1 necessarily only one, occuring in the 
last two rows and the first two columns of li), then in each equation there is at most one 
nontrivial term possible. In the first equation this occurs when A- 7 is obtained by adding a 
box to A' in the position of the top left box of the 2 by 2 square. In the second equation 
this occurs when \i % is obtained from \i by removing the bottom right box of this 2 by 2 
square. These two possibilities are easily seen to give the same value of x- So they cancel 
each other in (2.1.1) and give the desired triviality of x(Vz(/u), Vz(A)). 

(B.2) If n/v does not contain a 2 by 2 square, it must be a disconnected skew hook. 
It is easy to see that the only way to get a nontrivial term in the recursion is when [ijv 
has two connected components and one of the components is a single box. In this case 
each equation has exactly one nontrivial lower term and again these two terms cancel each 
other. (In the first equation get A J by adding a box to A' in place of the isolated box in 
fi/v. In the second equation get \i % by removing the same isolated box from //.) 

Case C. Suppose Xjv is a disconnected skew hook. Proceeding as before, we will 
analyze the ways in which one of the equations in (2.1.2) could yield a nontrivial lower x- 

(C.l) If X'/v is a disconnected skew hook as well, then it is easy to see that the only 
way one could get nontrivial lower terms in either equation is when the all of the following 
three conditions hold. X' jv must consist of exactly two connected components. Moreover 
the gap between the components must be exactly one box, i.e., adding just one box to 
X'/u, say in a-th row and 6-th column of A', should make it a connected skew hook. And 
\ijv must be a connected skew hook. In that case there is exactly one nontrivial term in 
each equation and once again these cancel each other. (A J is obtained by adding a box to 
A' in a-th row and 6-th column. In the second equation obtain /j? from [i by deleting the 
box in (a — l)-th row and (6 — l)-th column.) 

(C.2) So suppose now that X'/u is a connected skew hook (i.e., Xjv has exactly two 
connected components and one of the components consists of the last box in the first row 
of A.) Now if n/v fails to be a connected skew hook, it is easy to see that the entire analysis 
in Case B carries over. (After changing A to A' in the opening sentence of Case B, the rest 
applies verbatim.) 

(C.3) So we may suppose that n/v and X'/v are both connected skew hooks. The 
analysis here is entirely parallel to that in Case A with the following crucial difference. 
There is a fourth nontrivial term possible in each equation in (2.1.2). In the first equation 
one could get a A J by adding a box to A' immediately to the right of the right endpoint of 
A' jv. (In Case A this would just give A J = A, but here we get a lower term.) In the second 
equation one could get another \i % from [i by deleting the box immediately above the right 
endpoint of Xjv. (This is the putative fourth possibility discussed above immediately after 
(A.2.iii), which could not occur there.) Further, just as for the three pairs of possibilites 
in Case A, exactly one of this new pair of possibilities will actually take place. It is easy to 
see that the same calculation as in Case A gives the desired triviality of x(Vz(/i), Vz(A)). 

Clearly cases A, B and C exhaust all possibilities when the last column of A has 
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length 1 (in presence of the reductions that were made previously and which will be in 
force throughout the proof). This finishes Step 2. 

Step 3. (Further reductions.) If the last row of fx (which is absent from A by earlier 
reduction) contains exactly one box we will be done by Step 2 and conjugate symmetry 
(1.6.1). So henceforth we will assume that the last column of A as well as the last row of 
ix (each of which is entirely missing from the other partition) have lengths greater than 1. 

Now suppose that the first column of fx has exactly one more box than the first column 
of A. Then one reduces to the case in Step 2 as follows. In view of the discussion near the 
beginning of Section 2 we may take n (the rank of the defining representation of GL(F)) 
to be the number of rows in ix. Now by (1.6.2) one may replace the pair of partitions (ix, A) 
by the pair (fi c , A c ), which is covered by Step 2. So henceforth we will assume that the first 
column of \x has at least two boxes more than the first column of A, i.e., that the entire 
last two rows of fx are missing in A. By conjugate symmetry (1.6.1) we may assume that 
the entire last two columns of A are missing in fx. 

Step 4- (The last case.) Combining Step 1 and Step 3 we may and will assume the 
following. Each of X/u as well as ixjv contains exactly one 2 by 2 square and these squares 
are situated as described at the end of Step 1. In particular note that the last column of 
A contains exactly two boxes, as does the last row of fx. We will run the algorithm in 2.1 
with t = 2 and prove that x(^z(aO, Vz(A)) is 1. 

Consider all pairs of partitions (ix, A j ) (except (ix, A)) that occur in the first equation 
in (2.1.2). For such a pair to lead to nontrivial x, one of the two removed boxes in the 
last column of A must be added to the bottom of the first column of A'. This is the only 
way to remove the overconnectedness in Xjv and fx I v. So essentially one has to deal with 
placement of just one box. Similarly for a pair (ix l , A') to lead to nontrivial x m the second 
equation in (2.1.2), the second box in the last row of fx must be missing in fx 1 . Thus 
essentially one has to deal with removal of only one box in fx. Thus we have a situation 
very similar to the one handled in Step 2 and one can imitate the cases there. Instead let 
us take a shortcut. 

For future need let us separately treat the case when each of X/v and fxjv contains 
exactly four boxes, i.e., is a 2 by 2 square. Using preceding discussion, one easily verifies 
the desired result by an explicit calculation that is similar to but simpler than Case A in 
Step 2. So from now on suppose that each of Xjv and fxjv contains more than four boxes. 

Let £ be the partition obtained from A by removing the two boxes in the last column 
and adding one box at the bottom of the first column. One has 

x(VzM, Vz(A') ® A 2 (F)) = X (Vz(//), Vz(A)) x(Vfc(//), Vz(£) ® F) 

for the following reason. Recasting earlier discussion, any lower A- 7 that leads to nontrivial 
x(^z(aO, ^z(A J ) must contain £ as a subset. Thus, apart from A 1 = A, it suffices to consider 
only those A J that are obtained by adding one box to £. This is clearly a subset of the set 
of partitions £ fc such that Vz(£ fc ) is a filtration factor in a Weyl filtration of Vz(£) <8> F. In 
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fact the two sets will be the same unless the first two columns of A have equal lengths, in 
which case there will be exactly one extra £ fc obtained by adding a box in the last row of 
£. (This partition will not occur among the A- 7 since it amounts to adding two boxes to A' 
in the same row.) However in this case ///(/ifl will be disconnected since one of the 
connected components will be the two boxes in the last row and there must be at least 
three boxes in fJ,/(fi H thanks to the case treated separately in the previous paragraph. 
Thus for the extra one has x(Vz(/i), Vz(£ fc ) = 1. This proves the claimed equality. 

Now using (2.1.1) one has 

x(Vz(/0, Vz(X') ® A 2 (F)) = X (Vz(/i/l 2 ), Vz(X')). 
x(Vz(/*)> ^z(0 ® F) = X (Vz(»/l), K Z (0). 

We will be done once we show that the right hand sides of the preceding two equa- 
tions are the equal. To see this let 7r be the partition obtained by deleting the second 
box from the last row of \i. Recasting earlier discussion, any \i % that leads to nontrivial 
x(^z (/-**)) Vz(A') must be obtainable by removing one box from tt. Thus while calculating 
x(^z(/u/l 2 ), Vz(A')) it suffices to consider only those \i l that are obtained by deleting one 
box from n. We will set a up an almost pairing between the set S of such partitions and 
the set T of all partitions ft such that Vz(p) is a filtration factor in a Weyl filtration of 
Vz(/i/l). Suppose n l G S is obtained by removing a certain box from tt. Define a map / 
from S to T by sending \i l to the partition obtained by removing the same box from \i. 
Before examining the failure of / to be a pairing, let us note that 

x( W), Vz(X')) = x(Vz(f(»% Vz(0)- 

To see this one easily verifies pictorially that A'/ (A' fl /j, 1 ) = fl f{p?)) and that the only 
difference between ji 1 /{\' fl n l ) and fl f(fJ> 1 )) is in the arrangement of the three 

boxes at the left endpoint, where the former has the same shape as the partition (2,1) and 
the latter has the same shape as the skew partition (2,2)/(l). Clearly this difference is 
irrelevant for the formula asserted in Theorem 2.3. Further, the set differences between 
each pair of partitions have the same relative position. The stated equality follows by 
induction on the degree. 

Now let us deal with the failure of / to be a pairing. This can happen in two ways. 
(1) / will not yield the partition p, in T that is obtained by removing the second box in 
the last row of [i. However for this partition x(^z(aO, Vz(£)) is trivial since p,/(jl fl £) 
is disconnected in the last two rows. So we may ignore this failure. (2) /(//) will be 
undefined if the last but one row of \i consists of exactly two boxes and \x % is obtained by 
removing the second of these from tt. (Removing this box from \i will not give a partition.) 
However in this case x(Vz(/i l ), Vz(A')) is trivial since \i % /(/j 1 fl A'), which contains at least 
three boxes thanks to the special case trated above, is disconnected. So we may ignore 
this failure as well. This finishes the proof of Theorem 2.3. 

Remarks. 1) Certain products of the numbers on the right hand side of the formula 
in Theorem 2.3 have already appeared in the literature on Jantzen's sum formula. [JM] 
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shows that the determinant of the Gram matrix for a Specht module for the symmetric 
group is such a product. (I am grateful to Arun Ram for pointing out this fact and for 
supplying the reference.) 

2) Brenti's recent work on certain parabolic Kazhdan-Lusztig and .R-polynomials for 
the symmetric group also involves connected skew hooks, see [Brenti]. Theorem 2.3 and 
Brenti's work together suggest that at least for type A, Ext groups between Weyl modules 
should be somehow related to Kazhdan-Lusztig combinatorics. In the BGG category a 
connection between ordinary i?-polynomials for the Weyl group and Ext groups between 
Verma modules was suggested by Gabber and Joseph, but their guess was found to be false 
by Boe. Nonetheless in light of new evidence it seems that there may well be a relationship 
between appropriate .R-polynomials and Ext groups between Weyl modules. It would be 
very interesting to find a precise connection. 
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